perf(scan): reuse loaded global index planning context - #267

The failure came from process-wide BTree pages warmed by an earlier fixture. The cache key now includes the backend namespace, so cross-fixture pages cannot be reused. I restored the exact file-open assertion; all 12 primary-key sorted-index integration cases pass for Parquet and ORC. The latest fork workflows are waiting for maintainer approval.

Thanks for working on the repeated Global Index lookup performance.

Some downstream query engines, such as StarRocks, have a similar requirement. Their approach is to create a GlobalIndexReader through the public API in global_index_scan.h and retain/cache the reader for multiple queries. This already preserves the BTree reader's internal block cache without introducing process-wide cache sharing inside paimon-cpp.

The process-wide design in this PR introduces additional concepts and hidden global state, including the shared CacheManager registry, cache configuration LRU, cache_namespace, backend lifetime retention, and cross-reader allocator management. It also makes memory accounting and cache isolation harder for downstream engines that already have their own resource management.

Could we instead encourage the query engine to split the query into two explicit stages?

1. Global Index stage: create or reuse a GlobalIndexReader through GlobalIndexScan, evaluate the index predicate, and produce a GlobalIndexResult. This stage can be distributed by shard.
2. Data Scan stage: pass the returned GlobalIndexResult through ScanContextBuilder::SetGlobalIndexResult(), then use TableScan to generate the data scan plan and execute it.

The engine can cache readers using a key containing the table identity, snapshot/index generation, field, index type, shard/partition, row range, and relevant options. It should also ensure that the index lookup and subsequent data scan use the same snapshot, and respect GlobalIndexReader::IsThreadSafe() when sharing readers.

This keeps cache ownership, eviction, memory accounting, and isolation under the query engine's control while reusing the existing public API. Could we evaluate this approach before adding process-wide cache sharing to paimon-cpp?

Agreed. I removed the process-wide reader and BTree cache registry, namespace, and allocator-lifetime changes. The query engine now owns cross-query GlobalIndexReader reuse through the existing public API, pins the data scan to the same snapshot, and honors IsThreadSafe() around shared-reader evaluation.

This PR is now limited to reusing the schema, snapshot context, executor, memory pool, and caller-provided manifest cache already available inside one TableScan. The resulting diff is 4 files; paimon-global-index-test passes 132/132.